Electrical Engineering-Systems Dept.

*** SEMINAR ***

Yael Grossman

(M.Sc. student under the supervision of Prof. Tova Milo & Prof. Dana Ron)

on the subject:

CrowdMiner – Mining Algorithms From The Crowd

Crowd-based data sourcing is a new and powerful data procurement paradigm that engages Web users to collectively contribute data. Information is often gathered by posing questions to users, which they may answer for some small payment (such as Amazon’s Mechanical Turk), for social or moral reasons (Wikipedia), or even in the context of a game. A key property of the human knowledge is that it forms an open world; it is often the case that one wants to use the crowd to find out what is

interesting and significant about a particular topic, without having full knowledge about what the topic consists of. We are therefore not sure what sort of information we may be looking for, and consequently, do not know what questions to ask.

For classic databases, the problem of finding significant, unknown patterns in the data has been addressed via data mining. In particular, association rules, which capture when one set of data items indicates the presence (or values) of another set, were shown to be useful in identifying significant patterns. In this setting, the significant rules are assumed to be initially unknown, and data mining algorithms dynamically construct database queries to identify them. In a similar spirit, the question that we address in this paper is “Is it possible to mine the crowd?”

Following these observations, we present here the first crowd-mining algorithm, designed specifically for this context. We first define a formal model for crowd mining. Based on this, we design a novel algorithm for choosing the best questions to be posed to the crowd and mining significant patterns from the answers. We implement the algorithm and test its performance on benchmarks that we designed

for this purpose, demonstrating that it consistently outperforms alternative baseline algorithms.